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REPRODUCING SOUND ENCODED IN INFRARED INK ON PHOTOGRAPHS 

Field of the Invention 

5 The present invention relates to a camera isystem having an integral sound recording 

means for recording and outputting a permanent record of sound associated with a 
photograph and recorded in infra-red ink over the photograph. Hence, the present invention 
relates to a photograph reader to decode sound and play sound. 

The present invention also relates to a means for reproducing the recorded sound 
10 and discloses a photograph reader to decode sound and play sound. 

CO'PENDING APPLICATIONS 

Various methods, systems and apparatus relating to the present invention are disclosed 

in the following co-pending applications filed by the applicant or assignee of the present 
invention simultaneously with the present apphcation: 



International Patent Application Number 
(which would be filed in at a later date, when ttie number is 
recdved) 


Docket No. 


PCT/AUOl/ 


ART80 


PCT/AUOl/ 


ART81 


PCT/AUOl/ 


ART82 


PCT/AUOl/ 


ART84 


PCT/AUOl/ 


ART 85 



15 

The disclosures of these co-pending applications are incorporated herein by reference. 

Various methods, systems and apparatus relating to the present invention are disclosed 
in the following co-pending application filed by the applicant or assignee of the present 
invention on July 10, 1998: 
20 USSN 09/1 13,060, USSN 09/1 13.070 USSN 09/1 12,741 

USSN 09/1 12,785 and USSN 09/113.107 
The disclosures of this co-pending application are incorporated herein by reference. 

Various methods, systems and apparatus relating to the present invention are disclosed 
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in the following co-pending applications filed by the applicant or assignee of the present 
invention on June 30, 2000: 

PCT/AUOO/00743. PCT/AUOO/00744. PCT/AUOO/00745, PCT/AUOO/00746. 

PCr/AUOO/00747 and PCT/AUOO/00748 
The disclosurcs of these co-pending applications is incoiporated herein by reference. 

BACKGROUND OF THE INVENTION 

The production of images by means of photographic techniques have been well 

known for a substantial period of time. Further, recently, digital cameras have become 
increasingly popular where an image is captured by the digital camera device and stored for 
later printing. 

In applicant's applications USSN 09/113,060, 09/113.070 and USSN 09/113,222, a 
camera system has been proposed that includes an integral printer device for the printing 
out of sensed images. 

When using such devices and other image capture devices it will be desirable to be 
able to suitably deal with audio and other environmental information when taking a picture. 

In applicant's application USSN 09/113,107 one such device was disclosed in which 
audio data was recorded on the rear of a card or photograph using black ink on a white 
background. A reader could read the data and reproduce the signal through audio generator 
means when the card or photograph was passed through a scanner means. 

In US 5,896,403 (Nagasaki et al) a printing system is disclosed which prints 
images and audio information on print media, for example, a sheet, in dot codes which can 
be read by a pen device. The infonmation can be decoded notwithstanding scanning of the 
data being irregular or random due to the organization of the dot codes. In Nagasaki et ai 
system the scanning of data is low. In applicant's printing and scanning system as disclosed 
in USSN 09/113,107 the process is a pagewidth process rather than a line by line process as 
in Nagasaki et al 

SUMMARY OF THE INVENTION 

The present invention relates to a camera system able to record audio when 

acquiring a sensed image in addition to being able to playback the audio information when 
viewing a "photograph", the audio and "photograph" being recorded on the same surface of 
a print media. 
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In accordance with a first aspect of the present invention, there is provided a system 
for playing prerecorded audio encoded in a fault tolerant manner as an array of invisible ink 
dots printed on a photograph in substantially the same area thereas comprising: an infrared 
scanner means for scanning the printed invisible Ink dots of said prerecorded audio; a 

S processor means interconnected to said infra-red scanner means for decoding said scanned 
audio encoding to produce a corresponding audio signal; and audio emitter means 
interconnected to said processor means for emitting or playing said corresponding audio 
signal on demand , 

The invisible ink may be an infrared (IR) absorbing ink with negligible absorption 

10 in the visible spectrum. 

The encoding can include Reed-Solomon encoding of the prerecorded audio and 
comprises an array of IR (infrared) ink dots which are printed on the photograph using a 
page width ink jet printhead. The array of dots may be high frequency modulated to aid 
scanning, for example using a checkerboard pattern. 

15 The system can include a wand-like arm having a slot through which is inserted the 

photograph. 

In accordance with a further aspect of the present invention, there is provided a 
method of decoding a prerecorded audio signal printed as an array of invisible ink dots in a 
fault tolerant encoded form on a surface of a photograph coincident with an image printed 
20 on said surface of said photograph, said method comprising the steps of: 

(a) scanning the printed encoded form of said prerecorded audio on said photograph 
in the infra-red; 

(b) decoding said encoded audio signal; and 

(c) playing said audio signal on an audio output device. 

25 Brief Description of the Drawings 

Notwithstanding any other forms which may fall within the scope of the present 

invention, preferred forms of the invention will now be described, by way of example only, 
with reference to the accompanying drawings in which: 

Fig. 1 illustrates schematically the camera system constructed in accordance to the 
30 preferred embodiment; 

Fig. 2 illustrates schematically a printer mechanism for printing on the output 

media; 
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Fig. 3 illustrates a format of the output data on the photograph; 

Fig. 4 illustrates an enlarged portion of the output media; 

Fig. S illustrates a reader device utilized to read data from the photograph; and 

Fig. 6 illustrates the utilization of an apparatus of the preferred embodiment. 

S Description of Preferred and Other Embodiments 

The preferred embodiment is preferably implemented through suitable 
programming of a hand held camera device such as that described in USSN 09/113,060 and 
USSN 09/113,070. 

The aforementioned patent specifications disclose a camera system, hereinafter 
10 known as an "Artcam" type camera, wherein sensed images can be directly printed out by 
an Artcam portable camera unit. Further, the aforementioned specifications disclose means 
and method for performing various manipulations on images captured by the camera 
sensing device leading to the production of various effects in any output image. The 
manipulations are disclosed to be highly flexible in nature and can be implemented through 
15 the insertion into the Artcam of cards having encoded thereon various instructions for the 
manipulation of images, the cards being known as Artcards. The Artcam further has 
significant onboard processing power implemented by an Artcam Central Processor unit 
(ACP) which is interconnected to a memory device for the storage of import data and 
images. 

20 In the preferred embodiment, the Artcam device is suitably modified so as to equip 

it with a microphone device and associated recording technologies. When a picture is 
taken, the opportunity is provided to record either the surrounding sound environment or a 
message associated with the image. The recorded audio is then printed on the photograph 
in an encoded format, the encoding preferably being of a highly resilient form. The 

25 recorded audio provides a permanent audio record associated with the photograph. 
Subsequently, a playback apparatus is provided for scanning the encoded audio and 
decoding this information. 

Turning now to Fig. 1, there is illustrated, in schematic form the preferred 
embodiment I which includes the arrangement as described in the aforementioned patent 

30 specifications wherein an image 2 is sensed via a CCD sensor 3 and forwarded to an 
Artcam central processor 4 which includes significant computational resources as described 
in the aforementioned patent specifications. The Artcam central processor 4 can store the 
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image in memory 5 which preferably comprises a high speed RAMBUS (Trade Mark) 
interfaced memory. The Artcam central processor 4 is also responsible for controlling the 
operation of a printer device 6 having a page widtli ink jet pririlhead 15 (see Fig. 2) for the 
printing out of full color photographs, eg. 7, so as tb provide for instant images on demand. 

5 The printhead 15 can print color images and has at least (4) ink jet nozzles for printing at a 
resolution of 1600 dpi. The four inks include an infra-red ink for printing data, and cyan, 
magenta and yellow inks for printing a color image. Such a printhead, but having the 
^ capacity to print with six (6) inks, is disclpsed in applicant's applications 
PCT/AUOO/00743, PCT/AUOO/00744. PCT/AUOO/00745, PCT/AUOO/00746, 

10 PCT/AUOO/00747 and PCT/AUOO/00748 using a monolithic construction. 

In the preferred embodiment, the camera arrangement 1 is also supplied with a 
sound chip 10 which interfaces via RAMBUS bus 11 to memory 5 under the control of the 
ACP processor 4. The sound chip 10 can be of a standard or specialised form and can, for 
example, comprise a DSP processor that takes an analogue input 12 from a sound 

15 microphone 13. Alternatively, with increasing chip complexities (Moore's Law), the 
functionality of sound chip 10 can be incorporated onto the ACP chip which preferably 
comprises a leading edge CMOS type integrated in circuit chip. It will be readily evident 
that many other types of arrangements can be provided which fall within the scope of the 
present invention. The sound chip 10 converts the analogue input 12 to a corresponding 

20 digital form and forwards it for storage in memory 5. The recording process can be 
activated by means of the depressing of a button (not shown) on the camera device, the 
button being under the control of the ACP processor 4 otherwise it can be substantially 
automatic when taking a photograph. The recorded data is stored in the memory 5. 

Turning now to Fig.2, the camera arrangement preferably includes a printer device 6 

25 which includes printhead 15. The printhead 15 is utilized to print an image on print media 
17 and at the same time print audio (or other) information on the same face of print output 
media 17 using an IR ink. Similar arrangements but for printing information on the back of 
an output photo image are described in USSN 09/112,741, the contents of which are hereby 
incorporated by reference mutatis mutandis. By using a^ single printhead a simpler 

30 apparatus is provided than as therein disclosed. The applicant has disclosed an infrared ink 
suitable for this purpose in Australian Provisional Patents Applications PQ9412 and 
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PQ9376 both filed on August 14, 2000 and applicant's applications PQ9509 filed on 
August 18, 2000, and PQ9571, and PQ9561 both filed on August 21, 2000. 

Turning now to Fig. 3, there is illustrated an example of output printed on the 
photographic media 17. The media 17 includes a photographic image 18 along with which 

S is printed information 20 and data 22. The detail of the image 18 is not shown for ease of 
illustration. The information 20 can include location, date and time data with the location 
data being provided by means of keyboard input or, ialtematively, through the utilization of 
attached positioning systems such as GPS or the like. The information 20 is presented in a 
viewable form. The data 22 is provided in an encoded invisible (but infra-red visible) form 

10 with the beginning and end thereof marked by target boundaries 23. The format of the . 
encoding can be many and various, however, preferably the encoding is provided in a 
highly fault tolerant manner so as to tolerate scratches, grime, writing, wear, rotation, 
fading etc. One form of suitable technology is the printing technology utilized in the 
construction of "Artcards" as described in the aforementioned patent specifications USSN 

15 09/113,060 or as disclosed in USSN 09/112,070 and USSN 09/112,785 or the co-pending 

applications Docket Nos PCT/AUOl/ (Docket No. ART80), PCT/AUOl/ 

■ (Docket No. ART81) and PCT/AUOl/ (Docket No. ART82) 

filed concurrently herewith. The encoding format relies heavily upon utilisation otReed- 
Solomon encoding of the data to provide for a high degree of fault tolerance. A portion of 

20 the data 25 is shown in schematic form in Fig. 4 and the data comprises an array of IR dots 
as printed by the printhead 15 of Fig. 2 which is additionally modulated by a high 
frequency "checkerboard" pattern 21 added to the data so as to aissist in sensing of the 
encoded data. The data 22, in this particular instance, can comprise sound data as well as 
image information 20. The disclosed methodologies as discussed in applicant's co-pending 

25 applications Docket Nos PCT/AUOl/ ______(Docket No. ART80), PCT/AUOl/ 

^ (Docket No. ARTS 1) and PCT/AUOl/ (Docket No. ART82) 

provide for the storage of about 6 Megabytes of arbitrary data on a photograph having a 
size of 4" X 6" (102mm. x 152mm.) for data printed at a resolution of 1600dpi. 

When it is desired to "play back" the recorded audio, the photograph 17 is passed 

30 through a reader device 26 which includes pinch rollers for pinch rolling the photograph 17 
past a linear CCD sensor device 27 sensitive to infra-red, the photograph 17 being 
illuminated by an infra-red source of complimentary characteristics. The sensor device 27 
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can be a suitably adapted sensor device as described in the aforementioned patent 

specification or in applicant's application PCT/AUOl/ (Docket No. 

ART85) specification.. Suitable sensors are fouliid in flat bed scanners or facsimile 
machines. The sensors of the CCD device are^provided with lensing and/or filtering 

5 arrangements to optimise their sensitivity and coverage to that of the photograph's size and 
the spectral characteristics of the IR ink. An ink iwith minimal response in the wavelength 
range 400-700mm and maximal response in the wavelength range 700mm-1200mm has 
been found suitable as disclosed in Australian Provisional Patent Applications PQ9412 and 
PQ9376 filed on 14 August 2000 and applicant's applications PQ9509 filed on 18 August, 

10 2000, and PQ957i;andPQ9561filedbn 21 August 2000, 

Referring now to Fig. 6, there is illustrated in schematic form the operation of the 
audio reader device 26 of Fig. S. The linear CCD IR sensor 27 is interconnected to a 
second Artcam central processor 28 which is suitably adapted to read and decode the data 
stored in infra-red on the photograph. The decoded audio information is stored in memory 

15 32 for playback via a sound processing chip 33 on speaker 29. The sound processing chip 
33 can operate under the control of the ACP decoder 28 which in turn operates under the 
control of various user input controls 33 which can include volume controls, rewind, play 
and fast forward controls etc. Importantly, the CCD linear IR sensor 27 and the ACP 
decoder 28 can implement the reading process as if the information were printed in an 

20 Artcard or ah alternative Artcaid format as described in the aforementioned patent 
specifications USSN 09/113,070 and USSN 09/112,785, or applicant's co-pending 

applications Docket Nos PCT/AUOl/ (Docket No. ART80), PCT/AUOl/ 

Pocket No. ART81) and PCT/AUOl/ (Docket No. ART82) 

It can be seen from the foregoing description of the preferred embodiment that there 

25 is provided a system for the automatic recording of audio associated with an output image 
so as to provide an audio record associated with a photograph. There is also disclosed an 
audio reader system for reading an audio output recorded on a photograph with IR ink. 

It would be appreciated by a person skilled in the art that numerous variations 
and/or modifications any be made to the present invention as shown in the specific 

30 embodiment without departing from the spirit or scope of the invention as broadly 
described. For example, the utilisation of more complex audio recording and playback 
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techniques such as stereo and B-format techniques. The present embodiment is, therefore, 
to be considered in all respects to be illustrative and not restrictive. 
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CLAIMS 

1. A system for playing prerecorded audio encoded in a fault tolerant manner as an 

■1/' 

array of invisible ink dots printed on a pliotograph in substantially the same area thereas 
comprising: 

an infra-red scanner means for scanning the printed infra-red ink dots of said 
prerecorded audio; 

a processor means interconnected to said infra-itd scanner means for decoding said 
scanned audio encoding to produce a corresponding audio signal; and 

audio emitter means interconnected to said processor means for emitting or playing 
said corresponding audio signal on demand. 

2. A system for playing prerecorded audio as claimed in claim 1 wherein the invisible 
ink is an infra-red (IR) absorbing ink with little absorption in the visible spectrum. 

3. A system as claimed in claim 1 wherein said audio encoding includes Reed- 
Solomon encoding of said prerecorded audio. 

4. A system as claimed in claim 1 wherein said system includes a wand-like arm 
having a slot through which said card is inserted. 

5. A system as claimed in claim 1 wherein said encoding includes a high-frequency 
checkerboard modulation of said array of IR dots. 

6. A method of decoding a prerecorded audio signal printed as an array of invisible ink 
dots in a fault tolerant encoded form on a surface of a photograph coincident with an image 
printed on said surface of said photograph, said method comprising the steps of: 

(a) scanning the printed encoded form of said prerecorded audio on said photograph 
in the infra-red; 

(b) decoding said encoded audio signal; and 

(c) playing said audio signal on an audio output device. 
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7. A method of decoding a prerecorded audio signal as claimed in claim 6 wherein the 
invisible ink is an infra-red (BR) absorbing ink with little absorption in the visible spectrum. 

8. A method as claimed in claim 6 whereimsaid audio signal is Reed-Solomon 
encoded before printing as said anray of infra-red ink dots on the surface of said 
photograph. i 

9. A method as claimed in claim 7 wherein said encoded form includes modulating 
said array of infra-red dots with a high frequency checkeitoard signal component. 
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